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A3STRiCT 

A three-phase Delphi technique was used to help 
aeabers of the Natioifal Association for Research in Science I'eaching 
(WARST) identify areas and priorities for research* In t^e first 
phase, aeabers were asked to noainate three areas of needed science 
education research. A total of 729 areis were noaiaated by 248 
respoitdepts and were categorized into 35 generic research statea?nts* 
Secondly # respondent js assigned a priority rating to ea6h stateF<=^nt s. 
Data froa these responses were analyzei-to establish central tendency 
and disoersioc: characteristics for- each ite^- The stateaents were* 
then rerated by each participant after being given the group response 
data* Analysis of the responses froa the third phase revealei a 
higher degree of consensus on nearly all iteas.* In teras of NAPSr 
aeaber responses, the top five stateaeats were: (1) Application of 
learning add. cognitive development theories to classrooa instruction; 
(2) jiTialysis of sClassrooa learning behaviors that facilitate science 
learuinq; (3) I3entify what elements are essential in translating 
both research and a^velopaent activities into classrooa practice; {^) 
Analysis of .strategies for *^cquisi t ion, retention, and transfer of 
problea solvin:? in student's; ?.nd (5) lientif ication and validation of 
strategies to assist preservice anfl inservice teachers in acquiring 
specific teaching skills. (Author) 
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In April of 1976 the ^ej>artnent of Science Education at the University 
of Georgia be;ran a st\idy to determine priorities for research in science 
ed c c ation . The study was done at the request of the Pesearch Cozmittee of 

the National Associa-n'ion for Research in Science Teac ling (NARST)^'. The 

-J 

*cc«naittee had been asked by rhe Executive Board of Wr5RST to coordinate the 
study. * ; 

The initial irroetus for a study of reseeirch priorities cane frpa a' 
request by officials at the National Institute of Education for a statenient 
from NARST that outlined the research tasks the nembers of the organization 
•believed to be svost irrportan't, ' However, the* need for establishing priorities 
for res€iarch goes beyond the interests of a federal agency in establishing 
research prj^orities. College and university science education groups and 
individual science educarion researchers periodically ne€^l to examine the 
type of research activities on which they propose to expend their personal 
or institu'^ional resources. 

The' purpose of this report is twofold. First* the procedures followed 
by the University of Georgia research grout in obtaining infortnation fron^ 
NARST aembers about their priorities for research are presented- The 



second purpose is to describe the results of the sury ey^ b^r presenting the 
categories of res^ 
rankings of them. 



categories of research proposed- by menbers of the organization and their 



^Mercbers of the comnittee included Stanley Helgtson, CbaAr (Ohio 
State University), Edward Smith (Michigan State Universrty), William 
Torop (Vest Chester State College), and Paul Koutnik (Illinois 
Institute of Technology). 



The Delphi technique vas selected as a means of establishing research 
priorities azaong the NAPST nerbership. Delphi is a cethod for obtaining 
feroup judgements on factual natters, which lack precise inforraation, or on 
values, for which inforrsation is a matter of opinion (Swei^ert and Schabacker 
i97U). Tlic value of Delphi in this study rested in its potential to estab- 
lish a set of science education research priorities from a large group of\ 
respondents with diverse opinions and values, 

1 The iterative structure of the Delphi technique is the mechanism 
which distinguishes it from other group decision-making ^processes (Skutch 
and.ifell).^ Participants offer their perceptions and respond in the light . 
of previous actions and feedback from a summary of the judgeme^-ts of all 

respondents in previous to\irAs, A^coiding to Swelgert and Schabacker (197^;), 

' ' • ^ i ' ^ . • 

the process of making successive ^judgement's with feedback reduces the 

variance in the responses of the respondents* -This convergence appears to 

be greatest cn the first round after feedback than on any other sqbs^cuent. 

rounds (Cyphert and Gant, 157 0), 

Procedure ; 

The format of the strady was shaped by the decision to utilize the 
Delphi technique. Obtaininr input from as many N/*i?.ST members as possible, 
tabulating that data, and rerumini? thi^ information to assist responders 
in further ^'"•fining their research priorities w^s the study objective. 
Meeting, this objective indicated use of Delphi in a three-phased study 'pro- • 
cedure. 

Bef ore T^rior it ies could be established on a series of research tof^ics, 
the topics theoselves needed to be identified, Jhis was accomplished in 



Phase I of the s-tedy. In May, 1976 each member on the NARST mailing list 
(H=780) received a descriptiorn of the study and a form requesting nomination 
of three '.^eedec. areas of research in science education*" Responses from 
2*43 persons listed '729 areas of research. Twenty- two forns of the 780 were 

r 

returned as undeliverable as addressed. 

Phase I ended with a categorisiation of the 729 research area nomina- 
tions. Vhen the ciajority of 'the Phase I npminations were received, generic 
statements were generated by listing the priorities contained in the 
individual nominations. Vfhen duplicate content occurred, nominations were 
grouped into cateeories where apprcpriare. When 4 ncmination did not fit 
an established generic category, a new category was written. The result of 
this analysis was a set of thirt3,»-f ive statements. A second review of the 
nominations was then iiiade by another person in which each nomination was 

. V ' 

categorized by using the thirty-five' statements of priorixy. There was 
more than a .20 agreement between these two raters. The resxilts o^ this . 
process are in Table I. The contrast in fx^equency in iiomiucii-ioTn is illus- 
trated in the column "py^^se I - Initial Nominations." The range of nomina- 
tions for each statement was fr^^n 1 to 59, 

In Phase II, the list of 35 gtT;iex'Lr staremenls *of needed research in 
science education was "^Iso mailed to ^the 780 persons on the lI/iRST mailing 
list. Directions for this fprm requested each person to rat^ each generic • 
statement on a 1 to 10 priority scale, with one, indicating high priority 
and ten low priority. The 35 statements were listed in a random order. A 
total cf 327 respo-ises of MA?ST mailing) were returned. The data were 

analyzed to ascertain the node, mean, standard deviation, and percentage of 

respondents selecting each item. A summary, of the mean and standard devin- 

I* 
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tion of these data 'are displayed in Table I.^ 



\ 

\ 



^ Since a snail nuaber of Phase II responses . were i-oceived whirh did 

not list th^ name df the respotfd^, further inquiry was m^de to match , 
response with responder. The number of unusuable responses in Phase II was 
six. • * ^ . - ' 

in Phase. Ill of the study, eacJi respondent from Phase. II (N=327) was* 
nailed a copy of the statistics frcn P?iase II »^ as' well—es th^ir personal 
'response sheet. IJirections asked that respondents " — reconsjp^? your 
respj^se- in relation to those of your colleagues \ and agada^-pidicate a 
priority of frota one to ten. Of the 327 who received inf<2r?iation in this 
phase, 209 persons responded 5*^%. , Data" from Phase III were analysed in' a 
similar. wi^y. to Phase II and a summary of that data is contained in Table I. 

. ' . DISCUSSION 

Establishing priorities within groups of professional -researchers is 
not a simple task. The problem. is particularly difficult 'when the alterna^ 
tives are numerous as in this^tudy. The Delphi technia"*' 7^^^ f;nit^h1o fuv 
securing maximum inpBt in establishing priorities. As rioted, initial . 
suggestions came from 2ir3 persons, nearly 33 1 of the NAP^T membership. 

The frequency of nominations is itself one way to establish priori- 
ties* The number 'of, sup.Festions for each j'esearch area can be seen as .an 
indication of. the importance attached to these areas. Numbers of votes may 
be deceptive, however, for there may be widespread but weak interest. Or 
individuals may not consider the many possible areas of research. The 
second round of a Delphi study compensates for both these deficiencies. 
All nominations are shared with all r^oup members who rate each according 
to degnee of Importance. It is interesting to note in Table i that the 
generic research areas with relatively few nominations (e.g. #7,8) were 



given high ratings in subsequent rounds. On the other hand, two areas with 
many nominations (e.g. ^^11, In) were given lower rarings. 

Another irpporrant str^ength of the Delphi techniques is its potential 
for, achieving consensus at Phase III. The results of surveying the NARS7 
nembership indicate that^there was nore agreement on the ratings after Phase 
IIJ than after Phcise II. The variance among the responses decreased from 
Phase II to Phase III for thirty-two of the thirty-five research areas, 
-only one case .(?f30) did rhe variartce increase £:fter Phase II. 

• ' A second interesting finding during Phase III was the chan-^e in mean 
ratings. The rating of the 24 highest-rated research areas in'creased during 
Phase III. 'At' the same time, the six lowest-rated areas were rated lower at 
Phase III than at Phase II. In addition to increasing agreement on, eacfi 
iteci, there appeared to be increa^^g polarization between high- anJ low- . 
rated areas. 

Examining the 35 research areas crea'teg an almost irres-istible urge 
to identify /'the iriost pressing need" or ''highest- priority" and to find 
conc^onalities a.T.on^ Mgh- or lov;-rated areas. However, the wisdom of doing 
either is debatable. The research areas may have been stated in generic 
terms so broa<^ that it would be difficult to determine, what triggered a ' 
response in a responder. Also, the measurement error in each case is large 
enough to prohibit distinctions between priorities oF adjacent items 
iS^ = .15), Of course, there is a sirnificant difference between items 
separated by several ranks in the lict. 

Recognizing the uncertainty in the ta'ik , a few observations are 
desirable. Perhaps the strongest supported generalization about the. 
priorities is tha^ the practicality and onse of a pplication of the research 



prcxiuct diminishes with -che lower raxed pric|i?il:ies. The nicre highly rated 

research areas were characterized by applying theory to teaching or to 

learning or by identifying strategies th^it facilitate teaching or leaning. 

Implicit in both these areas is the potential for changing practic^ in 

classroom teaching or in teacher education. Developing or testing theory, 

f 

developing Diaterials, and pxirsuing interests in specific populations all 
were ranked ^eiw in the list. 

This preference for practical or applied research is distinctive 'and 
may represent a change* in position ove2T~"^e last dozen yearns ^ Ah assessment 
of change is- not possible, however, because priorities of^the sixties can 
only be inferred from the writing of a few (Tyler, 1967: Cooley, 1961; 
Novak, 1963). This study can provide baseline data in 1986 for determining 
if priorities have changed in the last decade. Even though this study does 
not identify the top priority for research in science education, it does 
indicate which areas* are a^ high priority of the science education research 
comniunity — knowledge which should^ Kel^^ researchers determine how to devote 
their energies and resources in the future. 
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TABLE I ' - ' 
Science Education, Ranking of Research Categories 



Statement Order Phase III Phase II 

Based on ^-^^ Ranking X SD . X SD 

Application of learning 

and cognitive deVelc^ment ' . ^ ' • . 
^ t^heories^ tb^ classroom 

•instruction. , 2.5 1.6" 3.0 2.0 



Phase I 
(Initial ^Number 
of Nomination) 



21 



Analysis of classroom 
teaching bishaviors that y ' 
facilitate science 

learning. \ 2.5 1.9 



2.1 



Id^tify what elements ^ ^- ^ 

are essential in trans- > 
lating both research and 
development activities 

into classroom practice. 2.7 1.7' 3.1 2.1^ 

Analysis of strategies 
for acquisition , retention 
and transfer of problem 
solving (critical^ thinking 
or inquiry skills') in ' ' 

students. -2.7 1.9 -3.1 2.1 



^2 



31 



26 X 



Identification and valida- 
tion .of strategies t<^ 
assist preservice and 
inservice teachers in ^• 
acquiring specific teaching , 
skills. 

Relationship between ,motiva~ 
• tion, attitudes and per- / 
formance' (in both students 
and teachers). 

Identification and develop- 
ment of teacher education 
strategies (inservice and 
preservice} designed *to 
facilitate professional 
9 growth and concerns of 
teachers, including 
commitment to continued 
growth. 



2.9 2.1 



i.O 2.0 



3.0 2.1 



3.3 2.^2 



3.5 3.0 



3.5 2.3 
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TABLE I (continued) 



Statement Order 
Based on Final Fariking 

8. Identification and* valida- 
tion of teaching behaviors^ 
and instructional strategies' 
that facilitate student • 
self-concept, knowledge 

and attitudes. 

9. Identification .and valida- 
tion of ^specific learner 

, characteristics ^/hich • 
V . relate to successful 

achievement in science. 

10/ Definition and .validation of 
goals of science instruc- 
tion, *e.g., balance between 
process' and process objec- 
tives, philosophical and 
theoretical basis of science 
instruction > arlriculatidn of 
goals for students at all 
levels, K- 16. 

11. Identification of factors 
which influence "formation 
of attitudes in students. 

. e.g., value clarification 
environmental education, 
attitudes toward science 
and . technology . 

12. Needs assessment of current 
• , ' practices as a basis, of 

X " decision. making for the 
development of science 
curriculum aiid teacher 
education materials 

— at junior high or 

. middle school level 

13. Needs assessment of current 
practices as a basis of 
decision making for the 
development of science 
curriculum and teacher 
education materials 

— at elementary. level 



Phase III ■ Phase II 



SD 



3.1 2.2 



3.2 2.0 



3.2- 2:3 



3.5 2.1 



SD 



3.5 '2.3 



3.6 2.2 



3.7 2.^ 



3.3 2.0 3.6 2.1 



3.4 2.1- 3.7 2.4 



4.0 2.6 



Phase I 
(Initial Number 
of Nomination) 



21 



21 



59 



10 1 



20 



ERIC 



11 



Stajzement Order * 

Bas^d'^on Final rRanking 

IM. . Appl-vcation of learning 
and cognitive development 
, theories to concept 
. formation J 



TABLE I (coi^tinued) 
* Phase III 



SD 



3.5 2.1 



Phase II 



SD 



Phase I 
(Initial Nurab^ 
of Nomination) 



3.9 2.^ 



63 



15. 



16. 



17. 



Design of longitud^inal 
studies to identify what 
^ kinds of gai^s are impor- 

-tan-t -to 3 vPir^ioty of 

Student ^bpulations . 

Development and evaluation 
of Instructional materials 
which draw from ^'^^ inte- 
gi?ate -more fuxly sciences, 
social, 'poXences, and 
inal-^*2niatics.^ 

NeedS" assessment of current 

practices as a basis of 
.decision making for, the 

development of science 
-^urriculurc and teacher - 

education materials 



3.5 2.2 



3.6 



2.1. 



3,9 2:u 



3.9. 2.U\ 



27 



10 



. ~ at 'Senior high level 3.7 • 2.2 

18. Analysis of relationship . ; 
.between discipline (sxibject 

, . . matter), structure and 

cognitive structure of the 
learner*.'' ' . > ' 3.7 2.2 

> 

19. Ideni&fication and valida- 
tion of alternative " . ^ 
evaluation schemed .for 

teachers and pupils.' * - . ; . ^ ^-8 2.0 

20 1 Needs assessniant ^of current 

' practices as a* basi§ of ^ v 

decisioi) making" fc^r the 
development of science * 
curriculum and t^chei? 
'education materials ' 



U.O 2.i| 



lU 



— ar college level 3 .8* 

I. ' . 



U.O 2.5- 



ura 2.1 



if.V 2.6 



;5 



17 
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TABLE (..f::.t Inu--:?) 



(Tritlal Nu-hc 

iUs«d on frlrul • xnkir - v - >" C'' of Noir.Initi^n 

21* AMlyisla of deci*i'n% 
r«Iatf*<l to curricul iTN 
iiipl«iaer*tdit ion e,-., 

eviloatlnf curric^l^ 
Inclu^lirjr a:.aly%i'% 

of u\struct ict.i: r.-itf^T * il - 
in sut^ect c-^nXer.* jr^^^-^ 
for t e<tc r r -.:? e i 'je nt 1 .3 1 T ~ 

of instrurt Icnal >r;->t^'^ 



racers s f r i r * \- 1 

art 9 , e t c V - . . . . •* ^ ■ 

J7. t>«v«lo?>^*r:t rf ^I**r-A.s* i^^'- 
tp« la 1 ]^ rc b 1 ♦T'/s - ' . sr . , l:^- . 
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TABLE I (continued) 



Stat€P»cnt Qrdmr 

B«t#d on rjUtiiil fUnking 

28 • Construction of a theory 
of »ci«nc€ initructioD . 

29. AstMscMmt or ioipact of 
non^school •xocritncea on 
•tvid«ntt* knovledfe In 
scimc«» nathemtica, 
social studlas* etc. 

30 • Analysis of residua effect 5 
of th« mST supported 
curriculuBt dav^lopr^nts • 

31. Inf-uenca cf ;>olitf^^l ar^i 
tschno jagical p^essur^^ on 
sci^nca instruction, scl^^nce 
teac^«r needs manpower 
e.f. , public attitudes and 
science enrolln^nts . 



Phase III Phase II 



SD 



SD 



Fhasa I 

( Initial Number 
of Woisination; 



3 ' 



'3 2^ "^'2 "^.3 



19 



3 3 



32, Identif icat irn cf i:iArv^v^*=?- 
ment skills needed f^^ 
maintainir.^. a vixhl-^ 
le-irnlng clissroota <»nvi^or.- 
r »nt--e. r, * di^jclplire 
f^vDu^ in r . bo^ ^ V <»ep lr\f. 
sVllls. 

Description of c*.irr*»';r 
perception! of '*i-1dl#' 
scHool* >*ig,h i?QV>ci ..^n t 
college grad^oatfS aN-^ut 
the nature of li^-*-, -%ritt<^r 

and desirability of alter-^? 

str itef ics . 



Ih. Ikn^lTtis of factors whi^N 
character !ae the rel^icta'^t 
scie^iCf t'-'itcH^r* 

35. Coweparrisoci of goal* cf 
•cleir.ce te^chinf todav 
with tfics* of 10-:^ T^r^rs 
SCO as vis'^iwd a rmriety 
of society's s#cff.«ts--e»g:. • 
p«r#?rt»» teachers. 5tuid«e*.ts, 
sr)d teacher •d^acat crs* 
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